Large network data centers provide economies of scale, large resource pools, simplified IT management and the ability to run large data mining jobs. Containing the network cost is an important consideration when building large data centers. Networking costs are one of the major expenses; as is known, the cost associated with providing line speed communications bandwidth between an arbitrary pair of servers in a server cluster generally grows super-linearly to the size of the server cluster.
Production data center networks use high-bandwidth links and high-end network switches to provide the needed capacity, but they are still oversubscribed (lacking capacity at times) and thus suffer from sporadic performance problems. Oversubscription is generally the result of a combination of technology limitations, the topology of these networks (e.g., tree-like) that requires expensive “big-iron” switches, and pressure on network managers to keep costs low. Other network topologies have similar issues.